Evaluate Geometry of Radiance Field with Low-frequency Color Prior

Radiance field is an effective representation of 3D scenes, which has been widely adopted in novel-view synthesis and 3D reconstruction. It is still an open and challenging problem to evaluate the geometry, i.e., the density field, as the ground-truth is almost impossible to be obtained. One alternative indirect solution is to transform the density field into a point-cloud and compute its Chamfer Distance with the scanned ground-truth. However, many widely-used datasets have no point-cloud ground-truth since the scanning process along with the equipment is expensive and complicated. To this end, we propose a novel metric, named Inverse Mean Residual Color (IMRC), which can evaluate the geometry only with the observation images. Our key insight is that the better the geometry is, the lower-frequency the computed color field is. From this insight, given reconstructed density field and the observation images, we design a closed-form method to approximate the color field with low-frequency spherical harmonics and compute the inverse mean residual color. Then the higher the IMRC, the better the geometry. Qualitative and quantitative experimental results verify the effectiveness of our proposed IMRC metric. We also benchmark several state-of-the-art methods using IMRC to promote future related research.Comment: 20 page

Active Visual Localization in Partially Calibrated Environments

Humans can robustly localize themselves without a map after they get lost following prominent visual cues or landmarks. In this work, we aim at endowing autonomous agents the same ability. Such ability is important in robotics applications yet very challenging when an agent is exposed to partially calibrated environments, where camera images with accurate 6 Degree-of-Freedom pose labels only cover part of the scene. To address the above challenge, we explore using Reinforcement Learning to search for a policy to generate intelligent motions so as to actively localize the agent given visual information in partially calibrated environments. Our core contribution is to formulate the active visual localization problem as a Partially Observable Markov Decision Process and propose an algorithmic framework based on Deep Reinforcement Learning to solve it. We further propose an indoor scene dataset ACR-6, which consists of both synthetic and real data and simulates challenging scenarios for active visual localization. We benchmark our algorithm against handcrafted baselines for localization and demonstrate that our approach significantly outperforms them on localization success rate.Comment: https://www.youtube.com/watch?v=DIH-GbytCPM&feature=youtu.b

CancerUniT: Towards a Single Unified Model for Effective Detection, Segmentation, and Diagnosis of Eight Major Cancers Using a Large Collection of CT Scans

Human readers or radiologists routinely perform full-body multi-organ multi-disease detection and diagnosis in clinical practice, while most medical AI systems are built to focus on single organs with a narrow list of a few diseases. This might severely limit AI's clinical adoption. A certain number of AI models need to be assembled non-trivially to match the diagnostic process of a human reading a CT scan. In this paper, we construct a Unified Tumor Transformer (CancerUniT) model to jointly detect tumor existence & location and diagnose tumor characteristics for eight major cancers in CT scans. CancerUniT is a query-based Mask Transformer model with the output of multi-tumor prediction. We decouple the object queries into organ queries, tumor detection queries and tumor diagnosis queries, and further establish hierarchical relationships among the three groups. This clinically-inspired architecture effectively assists inter- and intra-organ representation learning of tumors and facilitates the resolution of these complex, anatomically related multi-organ cancer image reading tasks. CancerUniT is trained end-to-end using a curated large-scale CT images of 10,042 patients including eight major types of cancers and occurring non-cancer tumors (all are pathology-confirmed with 3D tumor masks annotated by radiologists). On the test set of 631 patients, CancerUniT has demonstrated strong performance under a set of clinically relevant evaluation metrics, substantially outperforming both multi-disease methods and an assembly of eight single-organ expert models in tumor detection, segmentation, and diagnosis. This moves one step closer towards a universal high performance cancer screening tool.Comment: ICCV 2023 Camera Ready Versio

PyPose v0.6: The Imperative Programming Interface for Robotics

PyPose is an open-source library for robot learning. It combines a learning-based approach with physics-based optimization, which enables seamless end-to-end robot learning. It has been used in many tasks due to its meticulously designed application programming interface (API) and efficient implementation. From its initial launch in early 2022, PyPose has experienced significant enhancements, incorporating a wide variety of new features into its platform. To satisfy the growing demand for understanding and utilizing the library and reduce the learning curve of new users, we present the fundamental design principle of the imperative programming interface, and showcase the flexible usage of diverse functionalities and modules using an extremely simple Dubins car example. We also demonstrate that the PyPose can be easily used to navigate a real quadruped robot with a few lines of code

Recurrent neural network reveals transparent objects through scattering media

© 2021 Optical Society of America. Scattering generally worsens the condition of inverse problems, with the severity severity depending on the statistics of the refractive index gradient and contrast. Removing scattering artifacts from images has attracted much work in the literature, including recently the use of static neural networks. S. Li et al. [Optica 5(7), 803 (2018)] trained a convolutional neural network to reveal amplitude objects hidden by a specific diffuser; whereas Y. Li et al. [Optica 5(10), 1181 (2018)] were able to deal with arbitrary diffusers, as long as certain statistical criteria were met. Here, we propose a novel dynamical machine learning approach for the case of imaging phase objects through arbitrary diffusers. The motivation is to strengthen the correlation among the patterns during the training and to reveal phase objects through scattering media. We utilize the on-axis rotation of a diffuser to impart dynamics and utilize multiple speckle measurements from different angles to form a sequence of images for training. Recurrent neural networks (RNN) embedded with the dynamics filter out useful information and discard the redundancies, thus quantitative phase information in presence of strong scattering. In other words, the RNN effectively averages out the effect of the dynamic random scattering media and learns more about the static pattern. The dynamical approach reveals transparent images behind the scattering media out of speckle correlation among adjacent measurements in a sequence. This method is also applicable to other imaging applications that involve any other spatiotemporal dynamics

High Circulating Follicle-Stimulating Hormone Level Is a Potential Risk Factor for Renal Dysfunction in Post-Menopausal Women

ObjectiveMenopause contributes to renal dysfunction in women, which is generally attributed to estrogen withdrawal. In addition to decreased estrogen level, serum follicle-stimulating hormone (FSH) level increases after menopause. This study investigated the association between high circulating FSH level and renal function in post-menopausal women.MethodsThis observational cross-sectional study included 624 pre-menopausal, 121 peri-menopausal, and 2540 post-menopausal women. The levels of female sex hormones were examined by chemiluminescence and indices of renal function were measured using a clinical chemistry analyzer. The post-menopausal women were grouped into quartiles according to serum FSH levels.ResultsRenal function progressively declined from pre-menopause to peri-menopause to post-menopause, which was accompanied by increasing serum FSH level. In post-menopausal women, serum creatinine level increased with increasing FSH quartile, which was accompanied by a decrease in estimated glomerular filtration rate (eGFR) (p for trend <0.001); moreover, the prevalence of declined eGFR (<90 ml/min/1.73 m2) and chronic kidney disease (CKD; eGFR <60 ml/min/1.73 m2) increased (p for trend <0.001). Even after adjusting for confounders, the odds ratios (ORs) of declined eGFR and CKD increased with increasing FSH quartiles in post-menopausal women. The ORs of declined eGFR (OR=2.19, 95% confidence interval [CI]: 1.63–2.92) and CKD (OR=10.09, 95% CI: 2.28–44.65) in the highest FSH quartile were approximately 2- and 10-fold higher, respectively, than in the lowest FSH quartile (p<0.05). After stratifying post-menopausal women by median age (61 years), the OR for declined eGFR for each FSH quartile in the older group was higher than that for the corresponding FSH quartile in the younger group.ConclusionsA high circulating FSH level is an independent risk factor for renal dysfunction in women after menopause. Additionally, aging may aggravate the association of high FSH levels with reduced renal function in post-menopausal women

Heritable epigenetic modification of BpPIN1 is associated with leaf shapes in Betula pendula

The new variety Betula pendula \u27Dalecarlica\u27 selected from Betula pendula, shows high ornamental values owing to its lobed leaf shape. In this study, to identify the genetic components of leaf shape formation, we performed bulked-segregant analysis (BSA) and molecular marker-based fine mapping to identify causal gene responsible for lobed leaves in B. pendula \u27Dalecarlica\u27. The most significant variations associated with leaf shape were identified within the gene BpPIN1 encoding a member of PIN-FORMED family, responsible for the auxin efflux carrier. We further confirmed the hypomethylation at the promoter region promoting the expression level of BpPIN1, which cause stronger and longer veins and lobed leaf shape in B. pendula \u27Dalecarlica\u27. These results indicated that DNA methylation at the BpPIN1 promoter region is associated with leaf shapes in Betula pendula. Our findings revealed an epigenetic mechanism of BpPIN1 in the regulation of leaf shape in birch, which could help in molecular breeding of ornamental traits